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Background: Meta-analysis is a statistical technique in which the results of two or more independent studies, with similar objectives, 
are mathematically combined in order to improve the reliability of the results. The outliers, which may exist even in random models, can 
affect the validity and strength of meta-analysis results. 

Objectives: The current study uses "random effects variance shift model" to evaluate and correct the outliers in performing a meta- 
analysis study of the effect of albendazole in treating patients with Ascaris lumbricoides infection. 

Patients and Methods: The study used data from 14 clinical trials; each article was composed of two groups, a treatment group and a 
placebo group. These articles compared the effect of single dose intakes of 400 mg albendazole in treating two groups of patients with 
Ascaris lumbricoides infection.The articles were published in a number of internationally indexed journals between 1983 to 2013. For both 
groups in each article, the total number of participants, the number of those with Ascaris lumbricoides infection, and the number of those 
recovered following the intake of albendazole were identified and recorded. The relative risk (RR) and variance were computed for each 
article individually. Then, using meta-analysis, the RR was computed for all the articles together. In order to detect outliers the "random 
effects variance shift model" and "likelihood ratio test" (LRT) were used. Adopting the bootstrap method, the accuracy rates for sampling 
distribution of the tests, which were used for multiple testing, were obtained and the relevant graphs were depicted. For data analysis, 
STATA and R software were used. 

Results: According to meta-analysis results, the estimate for RR was 2.91, with a 95% confidence interval of 2.6 to 3.25. According to the 
method used in this study, three articles (articles number 4, 7, and 12) were outliers and, as such, they were detected in the graphs. 
Conclusions: We can detect and accommodate outliers in meta-analysis by using random effects variance shift model and likelihood 
ratio test. 

Ke3words:Meta-Analysis; Albendazole; Oudiers; Random Effects Variance Shift Model; Ascaris lumbricoides 



1. Background 

In health and medical sciences, the same topic may be 
investigated in numerous studies that may sometimes 
reveal contradictory results. One way of achieving a con- 
clusive result is the formulation of a meta-analysis study, 
which mathematically combines and analyses the results 
of different studies to achieve a more reliable outcome. 
In this method, the results from a number of individual 
articles with similar objectives are combined and then 
are subjected to more conclusive statistical analyses (1). 
The main objective of meta-analysis is the application 
of a test with a statistical power greater than that of the 
individual studies. Nonetheless, it is essential to pay due 
attention to factors that may systematically affect the 
overall results when commenting on the validity and 
strength of the results in a meta-analytic computation. 



One of the factors that can negatively affect the validity 
of results in a meta-analysis computation is the presence 
of outlier, an element of a data set that distinctly stands 
out from the rest of the data. The most thorough method 
of identifying distant data in terms of outliers has been 
formulated by Hedges and Olkin (2). Numerous graphic 
methods have been introduced for the evaluation of un- 
usual samples, but these methods can just be used for fix 
models (3, 4). However, bearing in mind that there are 
mixed models and random models as well, it is better 
to investigate outliers through these models too (5, 6). 
Gumedze and Jackson (7) introduced methods of detect- 
ing and accommodating outliers in a meta-analysis work 
by a random effects variance shift model. Therefore, the 
current study uses Gumedze and Jackson's model to mea- 



Implication for health policy/practice/research/medical education: 

Outliers in meta-analysis can affect the validity and strength of results; hence, by using the method of this article, researchers can detect and accommo- 
date outliers in meta-analysis. 

Copyright © 2014, Iranian Red Crescent Medical Journal; Published by Kowsar Corp. This is an open-access article distributed under the terms of the Creative Com- 
mons Attribution license, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. 



AlaviMajdHetal. 



sure the degree or size of outliers in a meta-analysis of the 
effect of albendazole on patients with Ascaris lumbricoides 
infection. Ascaris lumbricoides is one of the most soil-trans- 
mitted helminthes (STH) in the world. It is estimated that 
4.5 billion individuals are at risk of STH infection (Ascaris 
lumbricoides, hookworms, and Trichuristrichiura) and as 
many as 1.2 billion individuals might be infected with As- 
carislumbricoides, with Ascaris lumbricoides, close to 800 
million with Trichuris trichiura, and more than 700 million 
with hookworms (8, 9). The majority of STH infected indi- 
viduals are children and the infections are an important 
factor contributing to malnutrition in this age group (10). 

2. Objectives 

The current study used "random effects variance shift 
model" to evaluate and correct the outliers in perform- 
ing a meta-analysis study of the effect of albendazole in 
treating patients with Ascaris lumbricoides infection. 

3. Patients and Methods 

The current study used data from 14 clinical trial ar- 
ticles, investigating the effect of albendazole in treating 
patients with Ascaris lumbricoides infection (11-24). The 
articles had already been published in internationally 
referenced journals from 1983 to 2013. The articles were 
first obtained through different sources like the internet, 
data banks, and internationally recognized journals with 
some special criteria indicated below and then were sub- 
jected to the relevant meta-analyses. We used the terms 
"albendazole" in combination with "trial" or "study", 
"ascariasis", and "Ascaris lumbricoides". Bibliographies of 
identified articles were screened for additional relevant 
studies. The other criteria such as sample size, age, di- 
agnostic method, and dosage were checked in selected 
articles. The patients under study had been matched in 
terms of age in the studies under meta-analysis. Besides, 
all the studies had used a similar definition for recovery, 
the same amount and frequency of albendazole (a daily 
single dose of 400 mg of oral medication), and a similar 
binary response variable for recovery versus nonrecov- 
ery. For each of the 14 articles, the total number of partici- 
pants, the number of those infected with Ascaris lumbri- 
coides as well as those recovered following the intake of 
albendazole (for each of the two groups), the effect size, 
and variance of the intervention were computed. Bearing 
in mind that each study was composed of both the alben- 
dazole and the placebo groups, the responses produced 
would follow a dichotomous variable. To compare the ef- 
fect of albendazole on Ascaris lumbricoides, the cure rates 
were used to compare two groups under study. The prior- 
ity index of the effect of albendazole as cure ratio in inter- 
vention group to the placebo group was considered the 
relative risk (RR). The effect size or RR is shown by 9; then 
test statistic has to be defined for the significance of the 
effect size. Test statistics is defined by Q = £ Wj (y ; - 9") 
2 in which W; = 1 / V, and 9" = S (WjYj) / S (W t ). Under the 



null hypothesis, for all the effect sizes, which were similar 
or symmetrical, the distribution of Q statistics was chi- 
square with K-l degree of freedom (25). To detect outliers 
in the data of current study, the random effect Variance 
Shift Outlier Model was used. As such, this method was 
used to detect and test the outliers. For meta-analysis, 
the STATA software was used and the R software was em- 
ployed to administer this method (26). A brief account of 
the method is provided below. 

31 Random EffectVariance Shift Outlier Model (RVSOM) 

Following Gumedze and Jackson (7), basic model on 
standard random effects for meta-analysis is as follows: 
y = |jl n + u + e(l) 

Where y is a n-vector of estimated treatment effects 
for the n independent studies, \i is the unknown overall 
treatment effect, l n , is a n-vector of ones, u is a n-vector of 
unknown random effects, u ~ N (0, z 2 I n ) where z 2 is the 
between-study variance, which is unknown, e represents 
residual errors with, e ~ N ( 0,R) where R = diag (8 2 V 8 2 2 . . ., 
8 2 n ). The elements of R, the study variances, are regarded 
as known. The variance-covariance matrix of (1) can then 
be written as var(y) = V= x 2 I n + R with the variance of the i 
th study treatment effect given as var(yj) = z 2 + 8 2 j. 

32. Extending the Random Effects Model to the RVSOM 

According to Gumedze and Jackson (7), the random ef- 
fects variance shift outlier model (RVSOM) for the ith 
study (which allows an inflated variance for the ith study) 
takes the form: 

Y=|jl n + S j d J + u + e(2). 

This adds an extra term Sjdj to model (1), where dj is the 
jth unit vector of length n, i.e. with value 1 in the ith posi- 
tion and zero elsewhere, and is an unknown random coef- 
ficient with Sj ~N(0,w 2 j) for co 2 j > 0. 

The subscript j indicates which study has an inflated 
variance. Model (2) has the form of a simple linear mixed 
model with 8j as a random effect with variance <o 2 y The 
variance-covariance matrix for the data under the RVSOM 
for the jth observation is: 

var(y) = co 2 j d f d': + V 

An extension of model (2), which allows different inflat- 
ed variances for more than one study, can be written as: 
Y=ul n + D, 8, + u + e 

Where I is a subset {1, 2,..., r) of studies considered to be 
outliers, D = [dj] is an n x r matrix containing entires of 0 
and 1, where an entry of 1 in the ith row and jth column in- 
dicates that study i has the jth of r inflated variances, and 
8] is a r x i vector of unknown random effects. We referred 
to this model as an 'extended RVSOM' (7). 

3.3 Administering the Random Effect Variance Shift 

Outlier Model 

At First, we used forest plot diagram to detect outliers in 
our data, then we entered the outliers detected in forest 
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plot in the RVSOM Model as the jth observation. Then the 
model was fitted to the data and the degree of to" 2 for the 
jth was computed; the larger size of co" 2 j, the more likely 
for it to detect as an outlier. The likelihood ratio test (LRT) 
was used to measure the size or magnitude of co 2 j. 

The null hypothesis was H 0 :co 2 = 0 against the alterna- 
tive hypothesis was H A Q^<n 2 j > 0 for a RVSOM for observa- 
tion j. Stram and Lee (27, 28) showed that the asymptotic 
null distribution of the test statistic for testing this type 
of hypothesis was a mixture of two chi-squared distri- 
butions on zero and one degree of freedom. However, 
Gumedze and Jackson (7) showed that for the RVSOM con- 
ditions it cannot be met; hence, following Gumedze et al. 
(29), we had to use a parametric bootstrap procedure to 
obtain the distribution of our test statistic. 

3.4. Empirical Distribution of the LRT Statistic and 
Multiple Testing 

Under the null hypothesis, when there are no outliers 
in the data, empirical distribution of the likelihood ra- 
tio test statistics by a parametric bootstrap procedure 
is as follow: 

Step 1. Fit the null model (1) to the data to obtain esti- 



mates u. * and x~ 2 . 

Step 2. Generate a new data vector from model (l) and 
estimates n * and t~ 2 . 

Step 3. Compute the likelihood ratio test statistics LRT 
j, j = 1,..., n, by fitting the model (2) to the simulated data 
and compute and save the order statistics of the set LRT j 
for j =1,..., n. 

Step 4. Repeat steps 2 and 3 R times. This step generates 
an empirical distribution of size R for each order statistic. 

Step 5. Calculate the 100 (l-a)th percentile for each order 
statistic for the required significance level o. The percen- 
tiles, using a = 0.05 and k = 1 for largest order statistic, k = 
2 for second largest order statistic, are shown in the plots 
given in the results. 

4. Results 

The data used in this study were taken from 14 clinical 
trial articles that investigated the effect of albendazole 
on patients with Ascaris lumbricoides infection; y^ indicat- 
ed the relative risk in the ith article. As the forest plot dia- 
gram (Figure 1) indicated, articles 4, 7 and 12 were differ- 
ent from the rest of studies. The results of RVSOM model 
are shown in Figure 2 and Table 1. 



Study 
ID 



1 

2 
3 
4 
S 
B 
7 
8 
9 
10 

11 

12 
13 
14 



Overall (l-squared = 93 4%, p = 0 000) 



1 

00217 



RF. (95% CI) 



% 

Weight 



9!8 (2 56, 37 31) 
1.(8 (0 92, 1 26) 
6<8 (2 85, 14.71) 
2524 (1 66, 382 79) 
3 (8 (1 90, 5 01) 
2 64 (1 94, 3 58) 
3000(1 95, 461 84) 
1988 (1 32,298 56) 
2 65 (1 91,3.67) 
2C3 (1.72, 2.41) 
21 00 (6 85, 64 35) 
2923(1 86, 460 35) 
1520(1 03, 223 37) 
2 K (2 04, 3 22) 
2.81 (2 60, 3 25) 



0 88 
683 
2 18 
0 23 
4 96 
11 38 

0 14 
035 
9 51 
37.18 

1 24 
0 16 
0 23 
2474 
100 00 



I 

462 



Figure 1. Forest Plot Diagram Used to Investigate the Effect of Albendazole on Patients With Ascaris lumbricoides Infection 
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Figure 2a shows the estimates co 2 j form the jth RVSOM 
and the next two plots, Figures 2b and 2c, show the cor- 
responding estimates of the between study variance 
and the treatment effect. The plot 2d shows the likeli- 
hood ratio statistics from which we see that observa- 



tions 4, 7 and 12 are clearly detected as expected outli- 
ers; in particular, its LRT statistic is around three times 
the threshold for the first order statistic. All these fig- 
ures refer to the fact that articles 4, 7 and 12 had served 
as outliers. 



Figure 2. Random Effect Variance Shift Outlier Model Statistics Plotted Against Study Number for the Effect of Albendazole on Patients With Ascaris 
lumbricoides Infection 
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(2a) Variance shift estimates. (d 2 j; (2b) random effect variance estimates r" 2 ; (2c) estimates for treatment effect u"; (2d) likelihood ratio test statistics for a 



RVSOM for study j , plotted against study number. 



Table 1. Results From Clinical Trial Articles Investigating the Effect of Albendazole on Patients With Ascaris lumbricoides Infection 
(Al) a (11-24) 



Trial 


Authors 


Year 


Albendazole 






Placebo 










A1+ 


Al" 


A1+ 


Al 




1 


Oyediran and Oyejide 


1983 


22 


5 


2 


22 




2 


El-Masry etal. 


1983 


11 


0 


36 


4 




3 


Bwibo and Pamba 


1984 


36 


4 


5 


31 




4 


Ovedoff 


1984 


16 


0 


0 


12 




5 


Chien et al. 


1989 


37 


4 


12 


29 




e 


Upatham et al 


1989 


74 


4 


27 


48 




7 


Stephenson et al. 


1990 


7 


0 


0 


15 




8 


Sinniah et al. 


1990 


51 


5 


0 


10 




9 


Beach et al. 


1999 


61 


1 


23 


39 




10 


Olds et al 


1999 


179 


40 


92 


137 




11 


Patrick Pet al. 


2009 


63 


25 


3 


85 




12 


J. Ndibazza et al. 


2010 


9 


3 


0 


19 




13 


Speich B et al. 


2012 


9 


0 


0 


7 




14 


WiriaAE et al. 


2013 


144 


65 


64 


174 





a Al + refers to those recovered following the treatment and A.f refers to those who did not recover following the treatment. 
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Table 2. Estimated Parameters for Models Fitted to Invest^ 
tion a 


;ate the Effect of Albendazole on Patients With Ascaris lumbricoides Infec- 




Model M Q 




Model M , 




Parameter 


Estimation 


95% CI 


Estimation 


95% CI 


f 


11.247376 


(8.8,13.7) 


10.29457445 


(5.16,15.43) 


T 2 


20.083981 




85.23284239 










140.53558626 










302.27428826 




% 2 2 






0.01746554 





a Lumbricoides data: Overall treatment effect Variance shift estimates for jth the study (co 2 :), and between-study variance (t 2 ). M q : Random effects; 
model M Extended RVSOM for study 4,7, and 12. 



Table 1 includes the information of the articles used for 
meta-analysis, containing the names of the authors, year 
of publication, and the number of patients used albenda- 
zole regardless of its effect. 

Table 2 shows that the inferences were very robust when 
these three outliers were down weighted using the ex- 
tended RVSOM (which includes three co 2 j, j = 4, 7, and 12 
terms in a single model) and our findings greatly allevi- 
ated any concerns about the potential impact of outliers 
in these data. Once again, our proposed methodology per- 
formed well. 

5. Discussion 

The results obtained from meta-analytic evaluation of 14 
studies investigating the effect of albendazole on patients 
with Ascaris lumbricoides, detected articles 4, 7, and 12 as 
outliers. The articles under study were published in the 
internationally indexed journals during 1983 to 2013. The 
study used RVSOM to detect outliers. The results obtained 
from the tests and the relevant figures might indicate that 
the studies in rows 4, 7, and 12 served as outliers in this 
meta-analytic review and that the findings were largely 
reliable. Of course, with application of the method used 
in article 30, the articles 2, 10, and 14 were also detected 
as outliers. In this regard, in another research, a meta- 
analysis study was conducted to investigate the effect of 
albendazole on two groups of case and control patients af- 
flicted with Trichuris trichiura. The results indicated a rela- 
tive risk estimate of 2.06 with 95% confidence intervals of 
2.4 to 1.76. Therefore, with such results in mind, it can be 
claimed that, the cure ratio of individuals infected with 
Trichuris trichiura using albendazole was two times more 
than those who did not use the drug (22-24, 29, 30). In the 
current study, however, the relative risk estimate was 2.91 
with 95% confidence interval of 2.6 to 3.25, indicating that 
the ratio of individuals infected with Ascaris lumbricoi- 
des using albendazole was nearly three times more than 
those who did not take the drug. Therefore, it can be con- 
cluded that albendazole has been more effective in treat- 
ing Ascaris lumbricoides than curing Trichuris trichiura. 
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